ae at : 
ae 
ry : : ree 


e . ‘ rn 


-_ <i DOCUMENT RESUME 8 ° | = 

2 ~. oF 2 _ Bee : 

ED 179 766 °° | : = - CE 023 474, be 

7 ae - % . a ROO 

f AUTHOR - Bhaerman, Robert D,- . 0 tee «4s 

TITLE What "First Generation" Research on.Career Education - 

‘ —s Says to the Classrdéom Teachers--and Vice Versa. a 

‘ F  -Information'Series No. 179, a oe 4 
INSTITUTION ° EPIC CIearinghouge on Adult, Career, and Vocational. . 


Z Education, Columbus, Ohio. . 
SPONS AGENEY National: Inst. of Fducation \YDHEW), Washington, DVC.; 
_ 3% *  .-'Qhio State Univ.,, Columbus, Natdonal Center for 
-, 3 Research in Vecational Education. a fae! 
"PUB DATE , "79 : a : ~ . & 


CONTRACT * 400-76-0122 
NOTE ‘ 30p. . 


AVAILABLE PROM National Center Publications, National center for 
. .' Research in Vocational Education,/ The Ohio State 


t. ' University, 1960 Kenny Road, Columbus, Ohio 43210 
($2. 35) a. 2 4 
of EDRS PRICE © MFO1/PCO2 -Plus Postaqe. \ 
DESCRIPTORS *Career Education: Curriculum Develepment ; 


Educational Accountabjlity: Educational Facilities; 
: *Educational Researché Evaluation; *Evaluation 
ae 0 Criteria: Evaluation Methods: *Fvaluation Needs; 
. — =e _ . Faculty.Develcpment; Intervention; Learning’: ~ 
Activities: Petformance Factors: *Program . 
ah ' Effectiveness: Research Designs ROSenEch Problems ; 
ae 4 | State of the Art Reviews . 
6B YDENTIFIERS *Information Analysis Products 


ABSTRACT ~ . . / 
* "Intended for:the classroom teacher, this review of 
first generation cateer educaticn research is based primarily on 
Edwin L. Herw's cémprehensive review and synthesis,, “Research in 
Career Education: The State of the Art" (ED 109 Up) Herr's major 
findings have been summarized in a list of statemeéNjts. For example, 
+wo of these statements are as follows: (1) successful programs 
tended to show considerable commurity involvement and the bringing 
% together dt students and work jna adults in some way, and (2) several 
_ studies indicated that parents -are still the most influen ti actor 
A in students" career development. Several key observations a pre 
presented regardjng the impact and nature of thé research conducted. 
_ The following are representative: attitudinal and affective outcomes, 
a such as changes in self-cencept, were achieved ‘less ccmmonly than 
- cognitive content; there was little evidence that institutional 
effects were ‘considered; and in many projects the ocnceptual 
framework was obagfre, *hvs making it difficult to know what changes 
fin students or in™teachers could be anticipated. The remainder of 
thig document focuses on commen®s and quest¥ons teachers might want 
to,consider and to ask researcheré&, These questions are based on . 
guidelines in “Ways to. Evaluate Different Types of Career Education 
Activities: A Handbook of Evaluation. Models" (Anita Mitchell, et 
‘al.). (BM) . 
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The Educational Resources Information Center on Adult, . 


Career and Vocational Education (ERIC/CE)} is one of . 
sixteen clearinghouses in a nationwide: information 
system that is funded by the National Institute of° + 
Education. One of the functions of the Clearinghouse 
-is to interpret thé literature that is entered in « 
the ERIC data base. This paper, needless to say, 


should be of particular interest to classroom teachers” 


. 


who are, or whdé, shortly will be, involved in career. 
\education. | He 4 ; 
\ 


\ . ‘ . i 
Thetprofession is indebted to Edwin L. Herr of The 
‘Pennsylvania ’State University and Anita Mitchell, 


sof*the Southwest Regional Laboratory for their scholar- . 
-ship.ain the preparation of the basic source material 


for this paper and to Robert’D. Bhaerman who wrote .: 
this version specifically for the target audience ; 
of:classroom teachers. Recognition also is due David 
Gardner, Boston University, and Delia Neuman, The i 
National Center for Research in Vocational Education, 
as well as to Dr. Herr and Dr. Mitchell, for their 
critical review of the manuscript prior to its final 
"revision. Cathy Thompson assisted in the editing’ | 
of the manuscript and Millie Dunning typed the final 
draft. _ SP 
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'- Intended for the classroom teacher, this review of 
first generation career education research is based 
primarily on Edwin L. Herr's'comprehensive review e 
‘“tand synthesis, Research in Career Education: The State _* * 
of the Art. (ED 149 177) Herr's major findings have ; 
‘ ; been summarized in a list of statements. For example, 
if two of these statements are as follows: (1) success-~ 
ful programs tended to show considerable community , ° 
‘involvement. and the bringing together of students 
and working adults in some way and (2) several studies 
indicated that parents are still the most influential “ 
° factor in students' career development., Several 
° key observations also are presented regarding the 
impact and nature of ‘the research conducted. The 
following are representative: attitudinal and affec~ 
tive outcomes sucH as changes in self-concept were 
\. achieved less commonly than cognitive content; there 
was little evidence that institutional effects were 
considered; and in flany projects the conceptual 
framework was obscure, thus making it difficult to 
know what changes in students or in teachers could 
be anticipated. The remainder of this document ; . 
focuses on comments and questions teachers might " 
want to consider and to ask researchers. Based on © 
the guidelines for evaluation found in Ways to Evalu- 
atk Different Types of Career Education Activtties: A Handbook . 
of Evaluation Models (Anita Mitchell, et al.), these 
comments and questions relate to research design, 
supplementary activities, career education facilities, 
indirect interventions, staff development, product / 
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development, and curriculum implementation. (BM) 
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‘tant directions for ,ssuch re 


We all should be greatly indebted to Dr. Ed’ Herr 

of The Pennsylvania State University, who last year 
"delivered" a new member\to the family of career’ 
education practitionérs.\ Dr. Herr has previously 

written widely in the | of career education and 

is recognized for his scholarship in the field. 

{See References. ) \ 


; \ 
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While there have been “other career education, summaries : 
written before, by and large they were relatively : 

minor compared to Herr's cohprehensive review and 

synthesis,” Research in Cdireer Education! The State of: 


‘the Art (1978). I call this paper first-generation | 


since I am certain there will be many more studies 
ang. review and Svateneece to follow. 

\ 
Dr. ‘Herr reviewed ‘career educayioH studies of the 
exrly and mid-1970s and suggestes a number of impor- 


yas 


and what career ,education research might be comcerned <- — 
with when it reaches maturity. . . 7 


We believe it ‘is essential for teachers to know what 
some of the pluses and minuses have been, even though 
not all teachers might be involved »in career education 
activitfies--yet. If all ‘indications are correct, - 
we have not heard” the last’ of career education! a 
Space prohibits us from reviewing all Dr. Herr"s ° 
findings, but we will look at a significant part. ~ 


\ vo 


earch .to take in the a P ae 
‘late 1970s and beyond. We felt that it would be - & 
“very useful foriteachers'to know,what has ‘been done. 


? 


’ WHAT THE RESEARCH SAYS X 


‘> MAJOR FINDINGS 


* 


t 


What were thé most important facts and findings of hese 
first generation studies? Obviously, there were lots of” 
bits afid pieces. We will have to be extremely selective 
and representative, therefore, in summifg.up Herr's com- 


prehensive review. (This does not mean that you shouldn't 


go back and read his paper in its entirety. You should! 


It has a lot more in it than we can report. ). 


: — a 
Let's start with a fact probably everyone knows; namely, 
that career education is a’dynamie concept. Since the 
early 1970s, it has begun, in Herr's terms, to "reset 


' the instructional gyroscopes" from K-12 gn. many schaqel 


‘districts. With the funding of new legislation--the’ os 


Career Education Implementation Incentive Act,'P.L. 95-207 


‘on top of the previous Educational Amendments of 1974, 
P. Liye 3- 380, the movement to implement what already noe 
a started will béedih in earnest. ~ 


“The fact that effectjv eageee cduescien approaches vary 
also emergés fronffter rt! 
involve infusion,‘i.e. 7 the integration ‘of’ careek edica- 
‘tion, concepts into™“traditional subject content. Some a 
involve multiple components, infusion, field trips, 
career centers, and resource, pétscns. Some concentrate. 


\on separate classes or'.on' caréercenters. . Since the 


separate contributiqns of each” ‘approach: often are hard 
to pinpoint, -gareer education is hard to Study as a 


. whole. - Lt is sometimes. difficult\to know what was or was 
not a career education treatment. 


* yx 


. Because of its relatively. recent birth,.’ career ucation 
research cannot yet be expected to come up: with a} clear 
picture of, long-term effects qn .the attitudes er skills Js 


of perSons involved in *it. However,’ in spite of its. 


‘short. life-span, career educati‘on research has yielded ay 


great deal of information. , Of course, if one demands 
the absolute criterion of SC renetnic experimentation, 


s Study. For example, some efforts 


4 a 


‘relatively little can be said about whether career educa- 
. tion "works." If oné is willing to accept less precise 
‘ approaches the following findings. can serve as a summary 
: . esis. : ; 
_of first-generation career edufation research: : 
is aes 
' pr . 3 aa 
Finding: Many state and local.cgreer education efforts ._| 
have been undertaken... Most of these‘ have’ been 
at the elementary and junior high school levels. 
; Career one. ge since 1971, has grown-each. 
. year in nearly every. state. | ° nd 
Finding: Most. of the funding has,been with local or 
state rather than federal money, although 
large amounts of the financial support ha e 
come through federal "flow-through". funds jn 
; vocational education,.funds for cooperative . 
‘research, funds for education of the handi- 
capped, and other state-monitored funds, such’ 
we as ESEA and Titles III and IV. 
Ms . , , é _ a 
4 Finding: Surveys of parents, teachers,: and students - - ’ 
7 -* about career education have tended with few. .- 
23 exceptions to be, positive.” Elementary school . 
teachers have’ been more enthusiastic than 
secondary school teachers. In some places and 
among some populations, confusion about goals-. * : 
has prevailed. Some professional and lay - 
; persons seem to view career education’ as a fad. 


ee, Figgiing: Although some career -education activities have 
. : taken place in about one-third of the country's 

~y school districts, relatively few instances of’ 

a. 4 comprehengive K+12 programs have been evident. 


Finding: Not much evidence is available concerning 
“ efforts specifically tailored’ t@ spécial popu- : 

P : lations, i.e., to the physically or, mentally mS aa 
ae . handicapped, women, ethnic groups, and racial 
ao .* minorities. “With new federal priorities, this 

- situation appears to be changing. . j 

2 4 


Finding: There is little evidence that programs were te, 4 
pr, : developed from spe@#ific theoretical models or, 
ms 4 concepts. No resear¢h or evaluation studies 4 
ro 7 were found that directly pursued the relation- ‘ 
( ship of career development theory, and career ~ . 
e 


\ en 


Finding: 
Finding: 


3. Finding: 


a 


Finding: 
e j* 


Pinding: 


Z ” ; *‘ 
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“education. However, many caress 


.career education projects tended to be 


a a 


projects have apparently, borrowed their goals - 
from national or state demonstration projects — 


‘which may have originally used career develop- 


ment theory of some type. ° In eato be 


"atheoretical, " focusing on,means rather th 
ends. =_ . 5 


In a number of. projects, the term career seens 


.td have been misused. What occurred in these™ 
' projects was an emphasis on immediate choices 
“rather than dn shone or long- -term pranulng: 


Suceesetul programs have tended to. show con- 
siderable community involvement and the 


bringing together OK\students and working 


adults in Some Wey 
~ c 


Many evaluations. shave tended to describe the 
types and amount of participation by ‘teachers 
and students without assessing quality or the. . 
relationship of types of participation to : 
student learning. 


_ Several studies indicated that parents are =a 


still the most influential factors in the 


stufents'. career development. 


The results are’mixed on most career education 
Outcames.’ On balance, career education evalua- 
tions reported generally significant changes in 
knowledge of a wide range*.of occupations and 
in occupational awareness, motivation, work 
habits and attitudes, and attitudes toward 
school.., Sélf+concept changes, increases in: 
academic achievement, .and changes on several 
career development invehtories were less. 
certain. | Some projects achieved significant 
differences; others did not. Similarly, while. , 
goals were titled the,same way froiv project to, 
project, they frequent ly were assessed with 
different measures. Since these measures often 
were not correlated with each other, the 
various measurements were ‘hard to interpret. 
Re 


f a 
ae ? on 5 - - , * 


ao, ANS 120 0 _ 
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“ringing: on the pn fin 1 which eet ee a 
‘ interested, oe e.,. academic achievement, . sone ® 
., *studiés ‘suggested. statistically significant 
- dincreases in such achievement among students 
who’ had- participated ¢ in. career ‘education activ— 
_ gtties: However , other studies. indicated no —— 
Significant differences. between treatment.and, 
control groyps. -But there were no studies in 
. which sighificant results in academia achieve- 
ment favor&d a’ ‘on-career edueatidén group. In. 
’ short,*'the studies ,suggested that. pees as - 
education is likely to in¢rease the academic "' 
achievement of students; . however, when 
were not significant differences acrags 
and non-career education groups, it 
that career education did not affect a ademic . ar 
achievement in any nega CT ve way. 
t 
. Finding: The results. of research in exper ience- -based 
Jos career @éducdtion (EBCE) were not unlike those 
dn ' found in multi-component school-based pPRigrams. 
Academic achievement of EBCE students either 
increased or remained the same. Positive 
gains were reported for these students in such 
areas as career maturity, attitudes toward 
school, personal’ responsibility, specific. 
‘career, planning, Self-appraisal, and sense of 
achievement. Students generally enjoyed the 
individualized, flexible, and. community-based 
nature of these programs, although—tkey: some- 
times missed their friends and felt cut off: 
from their regular schools. Some parents felt’ 
that arate ai not sufficiently’ conerolted 
Or aiseipiane in the programs. - ae 


‘Several siuaies looked at the feasibility of 
infusing career education concepts,into consumer- 
education, reading, language, and other ‘content 
- areas. It was found that on measures of self- 

concept on the Iowae Tests of Basic. Skills, 

‘other measures of academic achievement, and in 
-gchool attendance, students exposed to career 

education did as well as or, in most studies, © 

better than students in twaditional classes. 
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‘Finding: ; “In the “several ‘sthajeh of ‘epebiittic abcisior- + ae eee, 

.* ¥ "making courses or short=term @hdividual or 9% 2 “FS. 

‘+. gYQup° vocatiohal counseling, positive results “’ : wet 
7 - generally were found. Th@-mpre specific the’ 


career planning experience to. which students «. 
swere exposed, the more posit ve the results he 
tended be. Students ‘in experimental career, = “. 
oe anning courses or counseling groups tended. 
‘‘to have higher academic achievement, more ee 
certainty about completing their-programs,, | °-. 
higher measured cereer gaeurity, and more... -' 
. skill at making appropr‘iate educatipnal,and + 
vocational choices .than students w were not 
y in these programs. peeve st was some 
indication that if styde learned about ; 
themselves* before they were exposed: to occupa- 
tional information, or if they requested such 
information_ whey they were ready, their learning 
was: facilitated een conventt j 
‘Finding: There were few evaluation. “studies of learning { 
, 5 resources, partitularly commercial onas. ‘he 
commercial materials available.tended to be . 
suppfements to rather than substitutes for the  -—. 
regular curriculum. Noncommercial and locally _ 
: developed materials tended to be ‘substitutes 
for regular curriculum materials. Very few 
commercial or noncommercial ‘materials were. 
available for special groups~*physically handi-~ 
capped, .ethnic or tacial minorities, senior 
citizens, women:or adults. ..Many of the commer- 
“N: cial materials and fewer noncommercial materials. 
were eck to-contain , racial and Bat stereo- ~*~ 
typical content. : 


ve 


4 4 ” 
Finding: “Studies of audio- visual material as well as of 
- career centers, tended to show that films or 
materials used independently of planned programs 
-. did not yyeld results as positive as those’ that 
octurred when these maferials were used in an 


se integrated program. “Tt. was found that the 
_. existenge of information. ‘was no guarantee of ‘ 
its use. — oo a! 


Finding; Studies of staff ‘development reflected a focus 
oa on inservice rather than i cl training. 
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i ree eee ‘The rasugts of some. stuigdeb suggested. gignititeank 
Oe he | aah Roe aififer'ences ‘in, vocational deve lopment far students oo 
” whose ,teachers had had various: types.of ‘inservice | ze 
- training sie eka ale students whose téachers had es 
’ : not: : ; 6 pS 
‘ Pg . , ' : d 
. These are some Of. the major findings of Dr. Herr' S review... 
a . and synthesis. - Readers are urged to Se of the ears ‘ 
; tions of dach finding in terms of thei 


own program. , 


ow 
‘ , 


SOME ‘REY OBSERVATIONS - 


2 4 eeanpeina “to assess the meaning of these findings is, 
complicated by the fact that career education, asfHerr . 
noted, is "not a singular process leading.to a singular 
result." Rather, it is a term used for a group of activi- 

. ties occurring at different educational levels 4nd pro- . 
ducing a number of outcomes. In many'.nstances, neither: "os - 
the approach nor the outcomes” of the aout were identi 
fied erearlys 

‘ y P ) * : 

The question most frequently daked by teachers and others 
is, “Does career education make any difference in student 
learning?" As the findings indicate, there now are some 
tentative answers, 


A more appropriate question, however, seems to ‘be, "What 

kinds of learning occur from which activities, for which 

students, and under what conditions?" It is clear that 

specific answers to this question are not yet available. 

Furthermore, some tentative answers that:are available are 

tainted by problems with the research desfign. 

woo : = / 
With such concerns in mind, the following observations . 
were made by Dr. Herr: 


Observation: Most of the findings about career education 
—e evaluation reports on funded Oj- 
ects wather Suioety ancien ate aoe research 
studies. there are several reasons: for this: 
career education's recency; the requirement 
for evaluative studies where project funds 

 .¢ , were allocated but restricting the types of 

local efforts to’ be evaluated; the small 
amounts of research funds available; t . 
relative lack of research which Spee . 


) 
ERIC | | 


fo, - ; . : : * as ms . eo 4 oft 
-: ° goes on as part of local efforts arr of 
: those stimulated by externa ie je a 


Observation: In many: projects’ the soneeutual framework 

was: ‘Obscure, thus making it difficult to ~ 
know. what changes in students or in teachers 
could ‘be anticipated. Also, the content of 
the career education processes a ae was 
not well described or assessed. 


Observation: ' Some project directors apparently used a 
published career education ‘assessment in- 
strument because ‘they had to evaluate the 
program rather than’ bécduse the instrument ° 
was appropriaté to project intentions. The 
result was that nortsignificant findings ° 
could mean gither that nothing was accom- 
plished or that ‘what was accomplished was 

. not measured. pee CCerver 


. 


Observation: It often was di seule to Sipe event bive ahi 
actual” changes made in educational or, counseél- 
; ing processes. In some cases, it appeared 
that some nonsignificant findings reflected 
the fact that nothing had really been done 
to affect the content or ae ae to 
which students were exposed. 
% “ 
Observation: In some studies where Significant Stee 
_‘ + were claimed ‘it was not clear that "statis- 
tical" significance could be equated with 
"educational" significance. In addition, 
two or three’ significant results out of 
forty or fifty tests in a particular project 
. were sometimes. given considerable credence 
. in spite of @ fact that they are likely 
° to be cha esults. 
Observation: There was ttle evidence ee 
effects on career education were considered. 
The role of administrative support, likely 
to be important in career education effec+ 
% tiveness, was ot given much attention. 
Case studies Bis paar a of the 
effects of. gertain administrative styles/ of 
resource levels, and of frequencies or forms 
of inservice training were not in evidence. 
r : _ ; 


Observation: Some career ‘education outcomes are easier 

- 4 - to ‘achieve than others. Attitudinal and | 
affective ‘outcomes, such ag changes in 
self-conceft, were attained less often o 
than cognitive content was mastered. a 


‘ 


Observation: One of the most troublesome aspects ‘of the 
; ~ evaluation studies was the tendency to ‘pool 
° - a variety of elements, describe them. collec ' 
. tively as career education, and contrast 
the results against so-called non~-career 
education comparisoy Qs: Where’ significant 
results were obtained ih, favor of career ° 
’ education, one had, no way of knowing which) 
“ specific elements made the most Significant 
‘‘dontributions to which behavioral changes. 
Thus, a whole series of reasons can be’, 
- effered to explain the, results obtained.  . . 
- The feeling persisted that "unnecessary 2 
things" were done in the name of. career 
education, but what these were was. not 
‘clearly determined. 


Observation: It seemed likely that jn some evaluations 
the. effects of some potent career education 
‘processes were cancelled out by other in-— 
‘effective processes. It also appeared that 
studying good teachers using gpecific 
treatments to obtain clearly stated objec~ 
«¢ tives would be a fairer ‘test of caree 
education impact than would combining good 
and poor or uncommitted teachers using non- 
descript techniques in the pursuit of un- 
, differentiated goals. 
Observation: Related to the matter oe "pooled effects" 
° was the lack -of knowledge about ‘the compara- 

' . tive advantages of different processes to 
accomplish particalar rea Far example, 
what are the advanYages of infusion over 
a specific course in decision-making skill? 
Do the results hold for populations of 


t different ages and at different grade. 
° levels? There was very eee i of this type 
of analysis. Complete descri 


ie of the 
jal so that 


bd 


activities studied — are essen 


a 


s 


“Q- ? 


; ee 


* 


£7 


: L . (others may -replidate research in different © 
ts “settings. 


2 7 4 groups have as much expectancy for. ae | 
as the career education groups, this lack - et 
oe of equivalence itself normally would be 
‘ . enough to explain the significant differ- 
es ences between the two. Of similar concern 
‘-is.the matter of "contamination" of com- ps 
parison groups. [It was not clear in many ye 
, ot of the reports whether experimental and 
comparison groups were the same. before 
career education was introduced to the, . 
former. In gome cases, also was not oy 
p 


. Unless non~caree sauewe ion comparison — 
a 


. _ clear whether Yontrol grdups received pe 
/ €" : information and other experiences similar [ 
, to those receiv d by the experimental group. 


Observation: It is difficult, ko be confident that career 
: education rather. "halo effect, 
maturation, or othe ations produced . 
the observed outcpmes. Also, outcomes in > 4 
on-going project were not netessarily 
} attained to the same degree each year. . 


Observation: "Perhaps too much has been expected too 
quickly. . Even though few studies met ideal ‘ 
experimental standards, a large numer of 
studies described career education's posi- 
tive impact. Given the fact that the bulk 
of these studies. occurred under natural * 
conditions and were conducted by relatively 
untrained evaluators, the results may be 
more impressive than they seem on first 
reading. Expérimental rigor aside, the 
( weight. of evidence favory career education. 
One additional observation: cee oyt, Director of . 
the Office of Career Education in the USOE, often has 
‘ indicated that critics of a new idea in education typically 
have called for definitive research results, even prior 
to the time the idea has. been fully developed. This Y 
appears to be the situation in career education and, 7 
throughout the years, likely has been typical of other’ 1 
Sauce clonal yy acters as weal. 
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‘The next stage ‘of carer education ‘xeseazch ‘would seem. 
: . 


"tant ones. 


.repearchers regarding research design. 


. ™ 
’ 
oe * ws 
t 


i) 


to involve formulating*and testing hypotheses aimed at- . 
discovering the hest-means of implementing career educa- a 
tiqn. If we are/to do\ this well, many of the observations - we - 


cited here need to be considered and acted upon. — £ ee 


It must be remembered that career education research is 
not static.’ New studies @re constantly appearing and, — 
as might be expected, they are generally improving in oo. oe 
design and implementation. As career education research 

moves to,greater maturity, that is, to the second. and. 
third generations, we will likely see more posifive and. ,° 
more comprehensive ‘results. oe Sarat 


o 
. #., 


SOME THINGS TEACHERS MIGHT LOOK FOR ee | _ 


Is there anything teachers cangdo to. assist in the vedacch 
progess? What can they say to’ or ask of researehers who 
will breed the-next generation of career education etudies? 


| 

t 
| 
| 
| 

Recently a very worthwhile new publication that has a’ ~ 

great deal of relevance to career education practitioners 

came to our attention. .As' indebted as we are to Dr.. err | 

for showing us where we ,have been, we are-.equally, grateful 

to Br. Anita Mitchell and her colleagues for developing 

a number of guidelines and models for evaluation of where 


‘we are. “(Ways to Evaluate Different Types of Career Educa- 


tion Activities: A Handbook of Evaluation Models, 1978) 
The handbook considered a number of issues that imply - 


_several things that teachers might say to future research- 


ers. Let us-take a brief ey of the most impor- 
q ; 
’ 6 


THE DESIGN OF RESEARCH 


Teachers might have these comments to make to future. 


a 


Comment: Before any research activity takes place, make 
os certain that the questions to be asked have 


° 


been formulated clearly enough to specify the 


“data requ ae ana ‘that the unde lying» 
gaaumpe tons and premises are clear. © 


* 


, * Comment : ‘Develop a conceptual framework for evgli ation 
b emphasizing short- or long-term goals’ and © 


8 e : dixect or indirect outcotes so that.the results 
off the evaluation can be understood in the con- 
tax f these goals” and outcomes , “ 


individuals. { "Did the program work?" rather 


7 - + Comment: Focys cuales and process rather than on © 
(than "Did the teacher do a soo sed 


« 


~ 
ry 


Comment: -Remember that one of the major reasons for evalua- 
tion is improvement. We should not. haye to wait 
a full year for results:only to find that ‘an 
f , activity is not working. We need:to know early 


e 


how a program is working and we need to'plan. 
corrections to make sure we, achieve the desired | 
goals. 


Comment: Be certain that we all- ce what we are cine 
” for. * Also, be certain that we have adequate 


resources to do the job effectively. ; - 
+ Comment: ' Be certain, too, enae we know which components 
‘ * will be evalu --and why. (The six suggested 
° An the handbo mre: supplementary activites, , 


‘Career education facilittes, indirect interven~ 
tions, staff development, product development, 
f and curriculum implementations.) *‘ a] ' 


Comment: Be, sure tO include us--the teacheys--in your 
oe - planning from the outset. Remember that evalua~ 
‘ tion will not work well if those who are affected 
: by it are not included both before the activity 
is implemented and during it. . 
I would also,.suggest that teache take Chie additional | 
comment: Please study very careffUully Ways to Evaluate 
; Different Types of Career Education Actsvitesa: A Handbook 
of Evaluation Models before you proceed any further with 
developing your research design! 
» 
é . 
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THE RESEARCH “ITSELF ‘ an) ese SP 


« ‘t The se ene are only a few pf thd genetal quesPions* 

. _* teachers might wish to ‘ask about supplementary activities, /. 
‘i.e. * field es i hénds—on activities, shadowing (foladWing 
or obsérving soWweone at work), supplemental curriculum : 

and guidance projects, -cafeer days, and career fairs: 


Questions: Were the seat arpa ae and implementation of. " 
these activities integrated with the’ regular _ o 
ak tae education activities to prepare a a 
students: for the supplementary activity? 
‘, to follow ub the experience? to enhance 
motivation for learning basic”skills? 


\ ; ae : 
Were the the a ‘tivity 
properly assigne \ adequately traindd? 


Was the planning of” the activity a coéperative. 
effort involving teachers, counselors, parents,. 
and community resource persons? , . “oy 


/ Were the resource® (tima,- money, materials) 
7 used to implement the ac ivity expended. 
g 3 efficiently and justifiable in terms of , 
potential ‘payoff? © oo a ae. 
: ae z 
t Sf Were counselors anditea hers 3ctive particypants / 
; in the activities?.-" { rs -) ie 


7 tg 


o 


: me . ‘ ° 
y | Did students gain in“eccup tional knowledge ° | 
. and improve their skiNs i decision making, 
. job hunting, ‘and/or jo ting as intended: 
i by the activity? Did they gain.in knowledge v 
\ of the educational requirements for the ‘ 
ie occupations covered by the activity? 
. _ Did stu ents ‘show inarkabied use of career 
educat i@f, facilithes a ‘result in one or 
: more basic academic courses or in the value 
. of school_generally? .Did they show more 
| positive attitudes toward work and/or increased 
desire a work? . : ; a 


é. ' . 

, . : , at 

7 a z aed -™. 
Pd . 
* . 
€ . 


vr os 
' t 
“Did some students benefit’ significantly. more "4 
i ae 2 less than others? ,What were the implications; . 
of °" . ‘for modifying ‘the ‘activity or for amproving o. a 
‘the pera of . students to it?. . . . ee : 
Tt e following are only a few of the general: questions a a 


teachers might wish to ask about cageer education oo. 
l.e., self- ~comtained Career centers, community career ~ ee. 
centers, or\ mébile vans, referral centers, such,,as dial-a- ’ 
ccareer-counkelor’ facilities, career centers housed within -~ 
classrooms; Sibrary-based progeams of informAtion | services?. af yo, 
and cates co nseling centers: 7 a a 2 at on 


hoicetione: What were the objectives of the facility and 1 
O we sa a they have been. ec camp ened?, a 
\ 
How weve student chan. asa result of using 
.the facility? 


How were the equipment and materials chogen . 
\ _ and what were they intended to accomplish? * , & 4 
8 J e™ 
What evidence have we that the resources are [ 
‘accomplishing the student growth objectives 
for which the fesources, were ' procured? ” . 3 


‘ ‘ é an ' 
7 ‘ 


{ 
Were the naceeie timely, acai ee, cahieces 


“What is the relevanee of the materials to the 

3 Bee and needs of the s@udents and the - ” : 

neds of and opportunities in the communat ys ” of 
Did students who completed a program in the. 
Career center: choose courses og study relevant ae 
to their career goals more frequently than 

“students not ee the progra 2 


Were students using the career centgr, more ; 
able than others to identify their strengths —~ 
and weaknesses and to relate these to : 
. educational and eeeapationes choices? — 
ft 
Were students ie pursued the career'‘center's . 
planned decision-making program more able - 
than others: to demonstraté a consistent *, 
decision-making process in actual situations? 


é 
av 


7 e 


4 


The fk Viewing: aretonly a féw~ of the “ggneral sieaetous:, sk 


teac 
i 


remight Wish to ask about indirect intefventions, — _ _ 


i.e., strategies and activities @esigned to affect vlocal. 
policy or state legislation; to = Bias fe publishers to + 4 
ia 


“remove sex, stereotyping and other bi 


role models; 
education. need; 


administrator training. ame to a cae » 


Smee in their prgrams. 


. Questions: 


_What were the reasons for success? (Consider 


factors from 


materials; to increase local bh training options; sd make - 2 
ir aos students full and factual occupational : cate 
{ informatton; 


to it\érease public awareness of .career 


to iRecsee pote to a wide range of: ieee 
d to motivate teacher, counselor,. ard 


t 

What actually happened?’ How did it as « - 
from what was ‘supposed to cig What wares. al 
the freasons for discrepancies? Were i } oe # 
plans implsthented as intended? 

Were there smeneteaeaees' problems’ in ON _ ga & 
“communication between the school and other 

qoote cee A _ a 

3 


"processes"--such as -communication, assistanqde, 
product delivery, and conflicts of values-\an 
"actors"--such as agencies, project staff 
members, and Communaey, members . ) 


Which, ‘agencies were most hel ful? Did any. 
gencies fail to cooperate?. Was it necessary 
to modify plans in order to reach the bp 
objective? 


$ 


kK ee a 


Did the change occuPe “Was the Sh enacted? 


Was the p Wey changed? Was the printed : 
material dified? Was the nature of the 
change. such aS reasonably to assure that it 
met #he need.,for which@t was ‘intended? 


. What, if any, were the“Significant side G 
eELeerss ; 


How -ma y students piaorited: from the. change? 
Does the chahge benefit various subpopulations 
différentiallly? How is student benefit 
evidenced? oes the benefit to students 
justify thegcost of the a ftivity> r 
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. Ge oth, Fe : owe ih | 
—" a “ * “y . _ Sota n. - tf Poe are ues 2 ° ” ypeces* 
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se a one Approadh more. rs Sfectivel “thai ang cher? ti eo 
ot yee a Ro | the“Berdefits! tov st dénits wax ant 5 suit, er 
WS ae of! ceat one) Anat ioretanal changes? | / a 
ae genre oe ae, : 
The following ; only. a “Few. ‘of the. erieral | questid a 
teachers mighf“wish’ to ask about be ger ,develapment ° ee ee cit 
‘activities, including internal staff’ ‘activities, such .as as 


‘sessions among - sdiiool | stQff; logally preparéd- staff ~ are ae 
. ‘development materials; .ex ernal/ educational res urces, 4 2g ae 
_ such’as professional: meetings, published staff ‘development = ss 
ttaterials, presentations By: state career. “educdtign experts ,° a, fF 
presentations by experts from, institutions’ of. highér.~- ° “So 
‘ learning, gummer ‘séasions- at colleges, and workshops by a et 
professional orgapizations ‘and ‘other eftefnal resourcesy . 


presentations by local: Ate st eressedatl ‘group! she ing igs 


such as visits t6 business7lahor/in usiry, work experience. ade 
in fields outside: eon On and woFkshops by: ‘private firms. ee, a 

; Guestdonscs was there a ‘ol@axly identified need:: ere —- 

: activity Aan 2 ase to meét? - a 


"it Did the: activity meet “Tes stated objectives? ‘e 


"Was the activity more . cost” “effective ‘than | eA 
dDther Greeunati ves uhder consideration? : 


e 8 


¢ s What were the pesrive SEreues Of. ~the activity? 


Thé following are only a ews of the eeneeetl agdeions 


teachers might wish to ask about product development, ea 
including textbooks, films, packages of writ xercises, ¢ 
audio tapes, packaged staff development programs, ‘Sets — 


of tests and evaluation snstruments, and evaluation ) % 
training handbooks. So ; 
Questions: Was the product implemented by users who were 

not under the direct control: of the developer? 

Was the product self-sufficient? Were all the 

"hugs" worked out of user instructrions, 

activity guides, and so fértie ae 
: Was the product sligcesa tui in producfhg. ° s & ae © 
desired outcomes? Were these produced with 

a minimum of undesirable side effects with - 

all members of the intended target euieeces 


- ae | ( 


o - 


~ 


“Were the. desdraéd .ov som a 
-appropfiate for” the Sntended audience and 


- ee eeu ae: use? .~ 


Were desired outcomes - pYodticed. aaa ar 
‘array. of circumstances yidkgce of effe How 


te” 


-genevalizable. was. ‘the evi ce of effects? = 
. a Vere desired owtcomes produced within | J a 
— .° S,applicable cost Limits and. Likely” ng tation : \ 
° a ee “ge | ; ait 4 a ae 
Bi ; fe / ‘ . a : a ; 
we Were ratiogen yee stable over “ttine? Can thd _ 


‘effects of ,th€ product be sustained over’ tim 


he event of: field testing for product os 
utibpzation, how many. i structional staff — 
wen involved and. yhat Were their roles? 
‘many. other per 
unseling personne r 
“community volunteers, and Sabeese ie on advisory | 
committees, were involved and what? were their 
rol¢gs?, “What were the essential characteristics,. 
e.g., Specialized training, or credentials, 
educational levels, sofio~economic, an 
emale/male dompositions of the persons 
involyed? neeB? thé participants Volunteers .. 
or conscri 
zo participation and/or attainment of desingd 
( oltcomes? W are secre time or .academica . 
credit provid : oe 
The :following are only ‘a few of. the. ‘enews questions 
teachers might wish to ask about curriculum tmpleméntation, 
including such strategies as special courses, separate 
units or topics in existing coursés, career education 
activities presented as_ enrichment options in existing 
courses, and career education: ‘concepts and activities used 
to s¥pplement basic subject matter in existing courses. 


How did curriculum activities differ among’ 
the groups of students involved in ¢areer 
education and from the activities of students: 
who received little or. no career education 
exposure? ST oes 


Questions: 


wf” 
v 


“ 


4g.,-guidance and |... 
mintatrators; parents, : 


>? Was there an incentive for. . e° 


. - “How é did the hp emented career etigatign? \- 
vee ss activities difflt*krom thoge originally: . gh 
+ designed’in staff training. workshops? © How" “-. 4. | 
and win what ways. we activities modified? . Cl ge 
In what! ways did «students behefit from - oe 
' gareer education activities? How did these _ 
benefits differ among the groups of students_ 
‘who participated? How did they differ from 
outcomes experienced, .by gtudents who, ‘received 
little er wp. cdreer educationp al 


. How dida career education a oe Pane 
‘to thé benefits in those situations in which ae 
differences among .groups were identified? . 
° we * 
_What are the: implicationg, for Pucuve: career. 
education curriculum acti ities? 


. | 


THE LAST WORD 
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‘Not long ago I completed a review-of thirty-eight career | 
education studies. In one of them, a researcher includéd 

A brief section. addressed to teachers. “, (The rece ee 
conducted by Clayton P. .OmVvig: The Effects of a gram 
of Caveer Education tin Kentucky's ‘Education Region IT. 4 
Phase If. Lexington: University of Kentucky, July 1976.) 
He had asked teachers several questigns, luding this: 
What was your most rewarding expertence in working with 
students? The teachers' résponses were‘along these lines: 
Because career education:established relevance in the 
‘classroom.and because*students could they¥ see how academic 
subjects would be utilized in later’ life, classes 
demonstrated greater eagerness about. their schooling. | 
Many teachers referred to increased student ieee and 
attendance and to the fact that student® were now given 
reasons for going to school and for setting goals. 
Students" concern for school s@emed to rtsult in an improved 
performance level. On@ teacher indicated that the most . 
rewarding experiénfe resulted from the students' - improvement 
-in grades and thefr changéd attitudes toward work. Another 

oe 


we e 
ge 


* “Phe students'. a¢ademic performance apparently did not 2 0 
€ 


we ' “ky 3 ee ai f 
f ; ee i . 
- Meith, 
, r 
: i 
* oe 
e 3 wey SS er « 
“a . ; t i ; su - a » @ . . a , ‘ 
teacher saw the year a¥. being the most rewgrding injall- Sp - 


" her’years of teaching. The teacher noted that the chiidren 
were happy, abSenteéism was‘at an all-time low, and achieve- 
ment was higher than.ever.before. This improvement in 


L 


preclude their growth in selfe-cénfidence and’ self-concep 
‘ areas that dre intrinsic to careér goed tg and that’ should «:. 


be inherent in any career education program. , i 
: ™ a : nee See eg ee 
These teachers—tand others “like them--are truly at thé heart (°° . 
of the career education process. Their judgments should | : 
_he ‘earnestly ‘regarded. Surely they should have; the last 
word, 7 a = { i , “ ‘ ‘ 
‘\~ i’ ‘ . ee ‘ is 
a _ @ : ‘ - . * . . ; é ! ; a 
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Herr, Edwin L. Research in Career: Education: The State 


of the Art. The National Center for Research -in Vocational 


Education, The Ohio State a aL Columbus, Ohio. - 


1977.; (ED 149 177), | 


Mitchell, Anita M., at ak hays to Evaluate Different 
Types of Career Education, Activities: A Handbook of 
Evaluation Models. Olympus Press, Salt Lake City, Utah, 
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Career Education ‘and Basie Academie Achievement. “A 
Descriptive Analysts, of whe Research. By Robert«D. -? 
Bhaerman. 1977. (ED 140 032) . os ue 


% 


» Descriptive analysis of thirty-eight studies 


/ from the 1970s, evaluating the.effectiveness 


. of career education programs focused on student 
achievement of competence in the basic académic 
skills. Provides appended data chart summarizing 
information on each study, including reséarch 
design,. instruments, regan aula ian 
findings, and conclusions. 
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What Does Career Education Do For Kido? i. Gyntnests 


-of 1975-76 Evaluatton Results. New Educational 
' «Directions, Crawfordsville, Indiana. 1977. (ED 143 831) 


Synthesis of evaluation results of forty- : 
five federally funded K-12 career education i 
programs. Organizes and tabulates diverse. —_ 
individual program goals for career education ; He 
released bY the U. pete of Education. 
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In addition, the reader might wish to explore the following 

representative samples of documents in the ERIC data base 

_ that deal with research and evaluation in career education: 


Career Edueattons . What Proof Dé We Have That It. 
‘Works? By Lois~eliin Datta and others. 1977. “te 151 516). 


Five papers summarizing “eyaluation efforts (¢ 

and results from career educution projects — 

in Texas, Kentucky, and Florida. Includes’ . 
results of individual programs and an overview 
of the evidence that indicates career: ‘education's 
ability. to progress, toward .six well- defined 
goals. 


A Review of Career Education Evaluatton- Studies. 


Monographs on*Career Education. By Thomas E. 
Enderlein. ee (ED 144 584) 4 


Summary of..the results of several evaluation 
studies of career education, with emphasis 

onthe specific areas of career awareness, , 
career decision-making, and .academic achieve- : 
ment. Examines ways tQ. determine program | 
effec#iveness, citing nine studies on attitudes 
of teachers, students, and. volunteers toward . 
{ qareer education. _ 


. 


‘The Effieacy of Career Education, National Advisory. 


Council for: Career Education. 1975. (Set of four: 
ED 121 953, ED 122 003, ED 122 ot ED 122 005) 


Series of papers addressing ave aeetve efforts. 

in four areas: (1) ¢areer awareness; (2) 
academic achievement; (3) career decision- * 
makingf and (4) career education projects, 
mainly those funded by the Vocational Education 

Act. Includes tentative conclusSions based , 
on study samples, and recommendations for . 
lee Diya 


Two other important studies’ by Dr. Herr gre as follows: | 
The Emerging.History of Career Education: Summar 


View. National Advisory Council or Career Education. 
1975. (ED 122 011) : 


Review and Synthesis of Foundatjons for Career Kducation® 
National Center for Research Pe wieeraee a Education. 


1972. (ED 059 402), 
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‘A number of programs exist for which evidente. of ° 


effectiveness was sufficient :to earn the JDRP ‘(Joint a? a 


Dissemination Review Pane] of the Department of HEW)?. 


"exemplary" 


stamp. 


In short, there are local evaluations 


that present evidence of effectiveness of | career 
education programs.’ 


The program activity title and jeeation are listed 
these JDRP programs: 


below for 


a: 


~ 


career Development Program 


Developmen 
Project 


— 


tal Career Guidance 


ay 


‘project CAP (Career, Awareness 


Program) - 


Project CDcC Gas Development 


ate 


¢ 


‘Centered Curriculum) 


ysProject CERES. {Career Education 


“BESpOne Ve 


to Every Student) 


Project, Equality we 


Project MATCH -(Matchiny Attitudes 
s to Career Horizons) 


gand aay 2 


For detail 
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Ben ants: . ‘ 
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” 


‘Pima County, Arizona 
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Grpe mana Retaneas 


Coloma, Michigan’ 


California 


4 


Seattle, Washington 


Ne ad : ; a 
Ontario, California 
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